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Abstract 

Diffusion state distance (DSD) is a metric on the vertices of a graph, 
motivated by bioinformatic modeling. Previous results on the convergence 
of DSD to a limiting metric relied on the definition being based on sym¬ 
metric or reversible random walk on the graph. We show that convergence 
holds even when the DSD is based on general finite irreducible Markov 
chains. The proofs rely on classical potential theory of Kemeny and Snell. 


1 Introduction and Main Results 

In 2013, Cao et al. j2j defined a metric (or rather, a sequence of metrics) on the 
set of vertices of a finite undirected graph, motivated by functionality consider¬ 
ations in protein interaction networks. In this class of metrics, called “diffusion 
state distance” (DSD), [2] considered a random walk on the vertices of the 
graph, and assessed the closeness of two states u and v by comparing the ex¬ 
pected number of visits to all states (within a given time horizon) when the 
initial state is u and when the initial state is v. (See below for the mathemat¬ 
ical details.) The following year, Cao et al. |3] extended the metric to graphs 
with weighted edges (representing confidence in the presence of the edge), which 
leads to consideration of Markov chains instead of random walks. The paper [3} 
also permitted directed edges. Further generalizations were considered in ra , in 
particular to bipartite graphs (which lead to periodic random walks). 

To proceed, we now describe a general framework for random walks and 
Markov chains on graphs. Let V be a (finite) set of points, representing the 
vertices or nodes of our network. For each (ordered) pair of vertices i and j, let 
Pij be a nonnegative number associated with the directed edge from i to j. (In 
[2 , the magnitude of /3y corresponds to the degree of certainty that this edge is 
present. If there is certainly no edge from i to j , then fyj is zero.) We consider 
the Markov chain on V with the property that when the chain is at a node i, it 
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chooses its next node with probability proportional to the weights on the edges 
leading out of i. That is, the one-step transition probabilities for the Markov 
chain are 

Pij = v ^ R for i,j £ V. (1) 

2-*,k£V P ik 

(We assume that there is no i for which the denominator in Equation ([1]) is 0.) 
The case of a classical random walk on a directed graph (with no weights) is 
obtained by requiring each /3y to equal 0 or 1, according to whether or not there 
is an edge from i to j. The case of a weighted undirected graph is obtained by 
requiring /3y = /3ji for every i and j in V ; this is the case assumed by Chung 
and Yao [J, which in turn was built upon by pQ. 

Note that the form of Equation © is completely general, since the transition 
probabilities ptj of any given Markov chain satisfy © if we let (3ij be p, : j for 
every i and j (since = 1 in any Markov chain). 

We shall denote the set of states of the Markov chain by V = {1 ,,n}. We 
shall write P for the (one-step) transition probability matrix with entries p^. 
For l = 0,1,2,..., we write pfj for the Z-step transition probabilities, which are 
the entries of the matrix P l . We say that the Markov chain (or P) is irreducible 
if it is possible to get from every state to every other state; that is, for every 
pair of states i and j, there is an Z > 0 such that pf) > 0. We say that an 
irreducible Markov chain (or P) is periodic if there is an integer d > 2 such that 
Pyj = 0 whenever l is not a multiple of d ; in this case d is called the period. We 
say that the chain is aperiodic if it is not periodic. 

Throughout this paper, except for the generalizations discussed in Section^ 
we shall assume that P is the transition probability matrix of a finite irreducible 
Markov chain. 

1.1 The Aperiodic Case 

In this section, we shall assume that P is aperiodic as well as finite and irre¬ 
ducible. 

Let 7r be the equilibrium distribution of P, i.e. the row vector (7Ti,... ,7r n ) 
such that 7rP = n and 7Ti = 1. (This is written n T in [2j and a in [6]). Under 
our assumptions, it exists and is unique. Moreover, the limit matrix 

W := lim P k (2) 

k—t oo 

exists, and Wjj = TTj for all states i and j. (This is called W in [2], and A in [5] 
and 0.) Writing 1 to denote the column vector with all entries equal to 1, we 
have W = Itt. 

For states u and v and any integer k > 0, we follow [2] and define 

k 

Pe (fc) (u,u) = 

1=0 
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Thus He^ k \u,v) is the expected number of visits to v within the first k steps 
of the chain, given that the chain starts at u. (This is called in ||6], and 
M u [y { u fc+1) ] in [5].) 


Proposition 1 For an aperiodic finite irreducible Markov chain: 
(a) The matrix Z defined by 


Z = ^(P-W) k 

k =0 

exists and equals (I — P + IT) -1 . (Note: Jl|/ writes D for P — W). 

(b) For all states u, v, and w, we have 

lim He^ k \u,w) — He^ k \v,w) exists and equals Z uw — Z vw . 

k—too 


Remark 2 Part (b) agrees with Lemma 3 of |IJ/, since Z uw — Z vw = — 

by)Zb w (where bi is the column vector having i th entry equal to 1 and all other 
entries equal to 0.) 

We now define the diffusion state distance metrics. For states u and v and 
integers k > 0, let 

n 

DSD {k \u,v) = £i* e^ k \u,w) — He^ k \v,w) |. (3) 

W=1 

Lemma 1 of |2| shows that DSD^ is a metric for aperiodic irreducible random 
walks; see Section [3] below for discussion and generalization. We also write 

f5D (oo) (u,u) = lim DSDW(u,v ) (4) 

k—± oo 


if this limit exists. 

Corollary 3 Consider an aperiodic finite irreducible Markov chain. Then the 
limit 0 exists, and 

DSD(°°\u,v) = ^ \Z UW -Z VW \. (5) 


Moreover, DSDis a metric. 

Corollary [3] was proven in |5] for undirected unweighted graphs (/5y = (3 ^ € 
{0,1}), and in [T| for graphs with symmetric weights (f3{j = (3ji). (Observe that 
the Green’s function matrix G in [1] is precisely our Z.) Both proofs rely on the 
diagonalization of the matrix P , which holds because the symmetry of weights 
implies that P can be expressed as a self-adjoint operator (equivalently, that 


3 


the Markov chain is reversible). In contrast, the proofs in the present paper 
rely on the classical potential theory for general finite (and countable) Markov 
chains in [5] and/or (BJ. 

We remark that Tj also considers the DSD^ metric for q > 1, defined by 
replacing the l\ norm on the right hand side of Equation 0 by the l q norm, i.e. 

/ n \^/q 

DSD^\u,v) = l^| He lk) (u,w)-HeW(v,w)\ q \ 

The extension of Corollary [3] and Equation ([4]) to DSD q is immediate. 

1.2 The Periodic Case 

In this section, we shall consider the case that P is periodic as well as finite and 
irreducible. An important example is the case of a random walk on a bipartite 
graph. This example was considered in Tj. 

As explained in Chapter 5.1 of [5], the relevant theory for aperiodic Markov 
chains extends to the periodic case with some modifications. There is a unique 
equilibrium probability distribution 7r such that ttP = 7r. We still define the 
matrix W by Wij = iTj. However, the limit of Equation 0 does not exist in 
the usual sense, but the equation is correct for the Cesaro limit. Moreover, the 
matrix I — P + W is invertible, and Proposition [Ha) holds if we interpret the 
infinite sum defining Z to be the Cesaro sum. 

For now, consider the general case that P is finite and irreducible (not nec¬ 
essarily periodic). For 0 < a < 1, let P a = al + (1 — a)P be the “lazy Markov 
chain”, which moves according to P except that with probability a it decides to 
stay wherever it is for one time unit. Observe that ir = irP a , so the equilibrium 
distribution is independent of a, as is the matrix W. Let Z a = (/ — P a + IT) -1 . 
Since this is well-defined, we can use Equation (0 as our definition of DSD(°°> 
for our periodic chain P , even though the limit of Proposition [T) may not exist 
except in the Cesaro sense. The following result shows that this definition is 
consistent with the usual definition. Observe that if 0 < a < 1, then P a is 
aperiodic (and irreducible), and so the results of Section fTTTl apply. 

Proposition 4 Assume that P is finite and irreducible. 

(a) For every a £ [0,1), we have Z a = (1 — a) -1 (Zo — aW). 

(b) Let DSD (°° ;a ) be the DSD (°°) metric of the Markov chain P a . Then for all 
states u and v, 

DSD(°°-’ a \u,v) = (1 -a^DSD&Mfav). 

In particular, part (b) generalizes Theorem 1 of [1J to general Markov chains. It 
shows that for periodic chains, the formal definition of Equation 0 is consistent 
by continuity with the (more directly motivated) definition 0 for aperiodic 
chains, i.e. 

lim DSD^ a \u,v) = DSD {oo ’°\u,v) forallu.u. 

O '—^0 
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2 Proofs of Results 


Proof of Proposition [XJ (a) This is direct from Propositions 9-75 and 9-76(4) 
of [6], or Theorem 4.3.1 of [5]. 

(b) This follows from Corollary 4.3.5 of (5\ Alternatively, here is a proof using 
the theory for denumerable chains (see discussion at the beginning of Section [3] 
below). Definition 9-24 of [6] defines the matrix C by 

Cij = lim (Ajf - Nf) ( = lim - He™(i, j))) 

K rOO \ K rOO / 

whenever this limit exists. Proposition 9- 770 of ® shows that C exists for 
aperiodic finite irreducible chains and that 

Cij = Zjj — Zij . 

It follows that 

lim He^ k \u,w) — He^(v,w) = C vw 

fc—too 


Cuw 

^VW ) Z U W ) 


□ 

Proof of Corollary [3] The second sentence of the corollary follows immedi¬ 
ately from Proposition [I] To show the final sentence, observe that Equation (j5j 
says that DSD^°°\u,v) is the l\ distance between rows u and v of the matrix 
Z. Since Z is invertible (by Proposition [lj a)), it must have distinct rows. It 


follows immediately that DSD (°°) is a metric. □ 

Proof of Proposition |4j By Theorem 5.1.3 of !5j, we have Z 1 = 1 and 
7r Z = 7r, where Z denotes either Z a or Zq. Therefore 

ZW = ZliT = in = W and (6) 

WZ = It rZ = Itt = W. (7) 

Therefore we have 

I = Z a {I - P a + W) = Z a ((1 - a)(I -P + W) + aW) 

= (1 — c^ZaZy 1 + aZ a W 


= (1 - o^ZaZo 1 + aW [by Eq. ©] , 

which yields 

Zq = (1 — a)Z a + aWZ 0 = (1 - o)Z a + aW [by Eq. 0], 

1 To explain the notation of [6] in this proposition: E is the matrix of all l’s, and Z^g is 
the diagonal matrix obtained from Z by making all off-diagonal entries 0. 
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Part (a) follows immediately. 

Next, recall the observation and notation of Remark [2] above. By this and 
Equation (|5]i . we see that part (b) will follow if we can prove that 7 Z a = 
(1 — a)~ lr yZ 0 , where 7 = — 6 ^. But this is a direct consequence of part (a) 

and the fact that 7 W = 7 l 7 r = 0 . □ 

3 Some Generalizations 

Since much of the potential theory for finite Markov chains in [5] also extends 
to countable chains [ 6 ], it is of theoretical interest to consider whether DSD 
theory extends to chains with countably many states. Our proof of Proposition 
|T] using results of [ 6 ] shows that this proposition extends to Markov chains that 
are strong ergodic (i.e. chains that are positive recurrent and have the property 
that for all states i and j, the expected square of the time to reach j [when 
starting from i] is finite); in [B], see Section 9.5, p. 274, as well as Definition 
9-71. However it is less clear whether Corollary [3] holds in this case, since even 
the convergence of the infinite sum is not obvious. 

Finally, we show that DSD^ is a metric for every finite or countable Markov 
chain except for some very special cases. For one counterexample, consider the 
irreducible Markov chain with states V = {1,2,3,4} such that P 12 = P 13 = 0.5, 
P24 = P34 = P4i = 1, and pij = 0 otherwise. It is not hard to check that 
He^ 2 \l,i) = He^ 2 \4,i) for every i, and so DSD^ is not a metric. Note that 
this chain has period 3. The next result shows that all counterexamples have 
features in common with this example. 

Proposition 5 Let P be the matrix of a (finite or countable) Markov chain, 
and let k be a finite nonnegative integer. Assume that there do not exist two 
distinct states i and j such that = p^ +1 ' ) = 1 and pfj > 0 for some t. 

Then DSD^ is a metric. 

In particular, if P is aperiodic or if the period of P does not divide k + 1, then 
DSD ^ is a metric. But the condition p^ +1 ) = 1 is much more restrictive than 
requiring the period to divide k + 1. Also note that Proposition [5] does not 
assume that P is irreducible. 

The following proof is essentially a streamlined generalization of the proof 
of Lemma 1 in [2]. 

Proof of Proposition [5j Write = 1 + P + ■ ■ ■ + P* for t = 0,1,.... 
Since DSD^ ( u , v) is the the l\ distance between rows u and v of the (possibly 
infinite) matrix , it suffices to show that all rows of N^ are distinct. 

Assume that u and v are distinct states such that rows u and v of N^ are the 
same. For vectors bi as defined in Remark^ let 7 = bf) — bfi. Then 7 = 0. 
Hence 'jN^P = 0. From the definition, we have N^P + I = N^ + P k+1 , 
and hence it follows that 

7 = lP k+1 - 
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Multiplying this equation on the right by b u gives the equation 1 = ( P k+1 ) uu — 
( p k + 1 ) vu . This can only happen if ( P k+1 ) uu = 1. Similarly, we deduce 
(P k+ i) vv = i Finally, since = Nm and > I vv = 1, there must 
be a t < k such that (P 4 )™ > 0. This contradicts the hypothesis of the propo¬ 
sition, and thus the proof is complete. □ 
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